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CLAIMS 



: is claimed is: 

A method for comparing a media sample and a media file, comprising: 

computing a set of sample fingerprints, each sample fingerprint characterizing a 

particular location within said media sample; 
obtaining a set of file fingerprints, each file fingerprint characterizing at least one file 

location within said media file; 
generating correspondences between said particular locations of said media sample and 

said file locations of said media file, wherein corresponding locations have 

equivalent fingerprints; and 
identifying said media sample and said media file if a plurality of said conesponding 

locations are substantially linearly related. 

2. The method of claim 1 wherein said particular locations within said media sample 
are computed in dependence on said media sample. 

3. The method of claim 1 wherein each sample fingerprint represents one or more 
features of said media sample near said particular location. 

4. The method of claim 1 wherein said sample fingerprints and said file fingerprints 
have numerical values. 

5. The method of claim 1 wherein values of said sample fingerprints specify a 
method for computing said sample fingerprints. 

6. The method of claim 1 wherein said media sample is an audio sample. 

7. The method of claim 6 wherein said particular locations are timepoints 
within said audio sample. 

8. The method of claim 7 wherein said timepoints occur at local maxima 
of spectral Lp norms of said audio sample. 
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1 9. The method of claim 6 wherein said sample fingerprints are computed from a 

2 frequency analysis of said audio sample, 

1 10. The method of claim 6 wherein said sample fingerprints are selected from 

2 the group consisting of spectral slice fingerprints, LPC coefficients, and 

3 cepstral coefficients. 

1 11. The method of claim 6 wherein said sample fingerprints are computed from a 

2 spectrogram of said audio sample. 

1 12, The method of claim 11 wherein salient points of said spectrogram 

2 comprise time coordinates and frequency coordinates, and wherein said 

3 particular locations are computed from said time coordinates, and said 

4 fingerprints are computed from said frequency coordinates. 

1 13, The method of claim 12, fiirther comprising linking a plurality of 

2 said salient pomts to an anchor salient pomt, wherein one of said 

3 particular locations is computed from a time coordinate of said 

4 anchor salient point, and a corresponding fingerprint is computed 

5 from frequency coordinates of at least one of said linked salient 

6 points and said anchor point. 

1 14. The method of claim 13, wherein said linked salient points 

2 fall within a target zone. 

1 15. The method of claim 14, wherein said target zone is 

2 defined by a time range. 

1 16. The method of claim 14, wherein said target zone is 

2 defined by a frequency range. 

1 17. The method of claim 14, wherein said target zone is 

2 variable. 
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1 18. The method of claim 13 wherein said corresponding 

2 fingerprint is computed from a quotient between two of said 

3 frequency coordinates of said linked salient points and said 

4 anchor point, whereby said corresponding fingerprint is 

5 time-stretch invariant. 

1 19. The method of claim 13 wherein said corresponding 

2 fingerprint is further computed from at least one time 

3 difference between said time coordinate of said anchor point 

4 and said time coordinates of said linked salient points. 

1 20. The method of claim 19, wherein said corresponding 

2 fingerprint is further computed from a product of one 

3 of said time differences and one of said frequency 

4 coordinates of said linked salient points and said 

5 anchor point, whereby said corresponding fingerprint 

6 is time-stretch invariant. 

1 21. The method of claim 6 wherein said particular locations and said sample 

2 fingerprints are computed from salient points of a multidimensional function 

3 of said audio sample, wherein at least one of said dimensions is a time 

4 dimension and at least one of said dimensions is a non-time dimension. 

1 22. The method of claim 21 wherein said particular locations are computed 

2 from said time dimensions. 

1 23. The method of claim 21 wherein said sample fingerprints are computed 

2 from at least one of said non-time dimensions. 

1 24. The method of claim 21 wherein said salient points are selected from 

2 the group consisting of local maxima, local minima, and zero crossings 

3 of said multidimensional fimction. 
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1 25. The method of claim 6 wherein said sample fingerprints are time-stretch 

2 invariant. 

1 26. The method of claim 6 wherein each sample fingerprint is computed from 

2 multiple timeslices of said audio sample. 

1 27. The method of claim 26 wherein said multiple timeslices are offset by a 

2 variable amount of time. 

1 28. The method of claim 27 wherein each fingerprint is computed in 

2 part from said variable amount. 

1 29, The method of claim 1 wherein said identifying step comprises locating a diagonal 

2 line within a scatter plot of said corresponding locations. 

1 30. The method of claim 29 wherein locating said diagonal line comprises 

2 forming differences between said corresponding locations. 

1 31. The method of claim 30 wherein locating said diagonal line further 

2 comprises sorting said differences. 

1 32, The method of claim 30 wherein locating said diagonal line further 

2 comprises calculating the peak of a histogram of said differences. 

1 33. The method of claim 1 wherein said identifying step comprises computing one of a 

2 Hough transform and a Radon transform of said correspondences. 

1 34. The method of claim 33 wherein said identifying step further comprises 

2 locating a peak of said Hough transform. 

1 35, The method of claim 1 wherein said identifying step comprises determining 

2 whether said the number of correspondences exceeds a threshold value. 

1 36. The method of claim 1 further comprising: 
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2 obtaining from a database index additional fingerprints characterizing file 

3 locations of additional media files; 

4 generating additional correspondences between said particular locations of said 

5 media sample and said file locations of said additional media files, wherein 

6 corresponding locations have equivalent fingerprints; and 

7 identifying media files for which a plurality of said corresponding locations are 

8 substantially linearly related, 

1 37. The method of claim 36 further comprising selecting a winning media file 

2 from said identified media files, wherein said winning media file has a 

3 largest plurality of substantially Knearly related corresponding locations. 

1 38. The method of claim 36 wherein identifying said media files for which a 

2 plurality of said corresponding locations are substantially linearly related 

3 comprises searching a fu-st subset of said additional media files. 

1 39. The method of claim 38 wherein additional media files in said first 

2 subset have a higher probability of being identified than additional 

3 media files that are not in said first subset. 

1 40. The method of claim 39 wherein said probability of being 

2 identified is computed in dependence on a recency of previous 

3 identification. 

1 41. The method of claim 39 wherein said probability of being 

2 identified is computed in dependence on a frequency of previous 

3 identification. 

1 42. The method of claim 38 wherein identifying said media files for which 

2 a plurality of said corresponding locations are substantially linearly 

3 related further comprises searching a second subset of said additional 

4 media files, wherein said second subset is searched if no media files in 

5 said first subset are identified. 
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1 43. The method of claim 36, further comprising ranking said additional media 

2 files according to a probability of being identified, 

1 44, The method of claim 43 wherein said probability is computed in part in 

2 dependence on a recency of previous identification. 

1 45. The method of claim 44 wherein said probabiUty is computed in 

2 part by increasing a recency score of a particular media file when 

3 said particular media file is identified. 

1 46. The method of claim 44 wherein said probability is computed in 

2 part by decreasing recency scores of said additional media files at 

3 regular time intervals. 

1 47. The method of claim 46 wherein said recency scores are 

2 decreased exponentially in time. 

1 48. The method of claim 43 wherein identifying said media files for which 

2 a plurality of said corresponding locations are substantially linearly 

3 related comprises searching said additional media files according to 

4 said ranking. 

1 49. The method of claim 36 wherein identifying said media files for which a 

2 plurality of said corresponding locations are substantially linearly related 

3 comprises terminating said search at a media file having a number of said 

4 substantially linearly related corresponding locations that exceeds a 

5 predetermined threshold. 

1 50. The method of claim 1 wherein said method is implemented in a distributed 

2 system. 

1 51. The method of claim 50 wherein said computing step is performed in a client 

2 device, said obtaining, generating, and identifying steps are performed in a 
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central location, and the method further comprises transmitting said sample 
fingerprints from said client device to said central location. 

52. The method of claim 1, further comprising repeating said computing, obtaining, 
generating, and identifying steps for sequentially growing segments of said media 
sample. 

53. The method of claim 1 wherein said obtaining, generating, and identifying steps 
are performed at periodic intervals on a rolling buffer storing said computed 
fingerprints. 

54. The method of claim 1, further comprising obtaining said media sample, wherein 
said computing step and said obtaining step are performed simultaneously. 

A method for comparing a media sample and a media file, comprising: 

receiving a set of sample fingerprints, each sample fingerprint characterizing a particular 

location within said media sample; 
obtaining a set of file fingerprints, each file fingerprint characterizing at least one file 

location within said media file; 
generating correspondences between said particular locations of said media sample and 

said file locations of said media files, wherein corresponding locations have 

equivalent fingerprints; and 
identifying said media sample and said media file if a plurality of said corresponding 

locations are substantially linearly related. 

A method for recognizing a media sample, comprising: 

computing a set of sample fingerprints characterizing a segment of said media sample; 
storing said fingerprints in a rolling buffer; 

obtaining a set of matching fingerprints in a database index, each matching fingerprint 
characterizing at least one media file and matching at least one fingerprint in said 
rolling buffer; 

identifying at least one media file having a plurality of matching fingerprints; and 
removing at least one sample fingerprint from said rolling buffer. 
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1 57. The method of claim 56, further comprising repeating said method for additional 

2 segments of said media sample. 

1 58. The method of claim 56 wherein said computing, storing, and removing steps are 

2 performed in a client device and said locating and identifying steps are performed 

3 in a central location, and wherein the method further comprises transmitting said 

4 sample fingerprints from said client device to said central location. 

1 59. The method of claim 56 wherein said computing step is performed in a client 

2 device and said storing, locating, identifying, and removing steps are performed in 

3 a central location, and wherein the method further comprises transmitting said 

4 fingerprints from said client device to said central location. 

1 60. A method for characterizing an audio sample, comprising: 

2 computing a set of reproducible locations in said audio sample; and 

3 computing a set of fingerprints characterizing said reproducible locations in said audio 

4 sample. 

1 61 , The method of claim 60, wherein said reproducible locations and said fingerprints 

2 are computed simultaneously. 

1 62. A program storage device accessible by a computer, tangibly embodying a program of 

2 instructions executable by said computer to perform method steps for comparing a 

3 media sample and a media file, said method steps comprising: 

4 computing a set of sample fingerprints, each sample fingerprint characterizing a 

5 particular location within said media sample; 

6 obtaining a set of file fingerprints, each file fingerprint characterizing at least one file 

7 location within said media file; 

8 generating correspondences between said particular locations of said media sample and 

9 said file locations of said media file, wherein corresponding locations have 

1 0 equivalent fingerprints; and 

1 1 identifying said media sample and said media file if a plurality of said corresponding 

12 locations are substantially linearly related. 
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1 63. A system for recognizing a media sample, comprising: 

2 a landmarking and fingerprinting object for computing a set of particular locations 
within said media sample and a set of sample fingerprints, each sample fingerprint 



3 

4 characterizing one of said particular locations ; 

5 a database index containing file locations and corresponding file fingerprints for at least 

6 one media file; and 

7 an analysis object for: 

8 locating a set of matching fingerprints in said database index, wherein said 

9 matching fingerprints are equivalent to said sample fingerprints; 

1 0 generating correspondences between said particular locations of said media sample 

11 and file locations of said at least one media file, wherein corresponding 

12 locations have equivalent fingerprints; and 

£3 13 identifying at least one media file for vdiich a plurality of said corresponding 

f| 1 4 locations are substantially linearly related. 

% 1 64. A computer-implemented method for creating a database index of at least one audio file 

\| 2 in a database, comprising: 

^ 3 computing a set of fingerprints representing features of each audio file, each fingerprint 

o 4 characterizing a particular location within said audio file; and 

f 5 storing witiiin a memory said fingerprints, said locations, and an identifier of each 

6 media file, wherein each corresponding fingerprint, location and identifier is 

O 7 associated in said memory. 

1 65. The method of claim 64, further comprising sorting said database index by 

2 fingerprint value. 

1 66. The method of claim 64 wherein said particular locations of each audio file are 

2 computed in dependence on said audio file. 

1 67. The method of claim 64 wherein each fingerprint represents at least one feature of 

2 said audio file near said particular location. 

1 68. The method of claim 64 wherein said fingerprints are numerical values. 
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1 69. The method of claim 64 wherein values of said fingerprints specify a method for 

2 computing said fingerprints. 

1 70. The method of claim 64 wherein said particular locations are timepoints within 

2 said audio file. 

1 71. The method of claim 70 wherein said timepoints occur at local maxima of 

2 spectral Lp norms of said audio file. 

1 72. The method of claim 64 wherem said fingerprints are computed from a frequency 

2 analysis of said audio file. 

1 73. The method of claim 64 wherein said fingerprints are selected from the group 

2 consisting of spectral sHce fingerprints, LPC coefficients, and cepstral coefficients. 

1 74. The method of claim 64 wherein said fingerprints are computed from a 

2 spectrogram of said audio file. 

1 75. The method of claim 74 wherein salient points of said spectrogram comprise 

2 time coordinates and frequency coordinates, and wherein said particular 

3 locations are computed fi-om said time coordinates, and said fmgerprints are 

4 computed from said frequency coordinates. 

1 76. The method of claim 75, further comprising linking a plurality of said 

2 salient points to an anchor salient point, wherein one of said particular 

3 locations is computed from a time coordinate of said anchor salient 

4 point, and a corresponding fingerprint is computed from frequency 

5 coordinates of at least one of said linked salient points and said anchor 

6 point. 

1 77. The method of claim 76, wherein said linked salient points fall 

2 within a target zone. 
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1 78. The method of claim 77, wherein said target zone is defined 

2 by a time range. 

1 79. The method of claim 77, wherein said target zone is defined 

2 by a frequency range. 

1 80. The method of claim 77, wherein said target zone is 

2 variable. 

1 81 . The method of claim 76, wherein said corresponding fingerprint 

2 is computed from a quotient between two of said frequency 

3 coordinates of said linked salient points and said anchor point, 

4 whereby said corresponding fingerprint is time-stretch invariant. 

m 1 82. The method of claim 76, wherein said corresponding fingerprint 

W 2 is fiirther computed from at least one time difference between 

^ 3 said time coordinate of said anchor point and said time 

''^ 4 coordinates of said Unked salient points. 

O 1 83. The method of claim 82, wherein said corresponding 

J 2 fingerprint is further computed from a prodixct of one of 

Q 3 said time differences and one of said frequency coordinates 

04 of said linked salient points and said anchor point, whereby 

5 said corresponding fingerprint is time-stretch invariant. 

1 84. The method of claim 64 wherein said particular locations and said fingerprints are 

2 computed from salient points of a multidimensional ftinction of said audio file, 

3 wherein at least one of said dimensions is a time dimension and at least one of said 

4 dimensions is a non-time dimension. 

1 85. The method of claim 84 wherein said particular locations are computed from 

2 said time dimensions. 
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1 86. The method of claim 84 wherein said fingerprints are computed firom at least 

2 one of said non-time dimensions. 

1 87. The method of claim 84 wherein said salient points are selected from the 

2 group consisting of local maxuna, local minima, and zero crossings of said 

3 multidimensional function. 

1 88. The method of claim 64 wherein said fingerprints are time-stretch invariant. 

1 89. The method of claim 64 wherein each fingerprint is computed firom multiple 

2 timeslices of said audio file. 

1 90. The method of claim 89 wherein said multiple timeslices are offset by a 

2 variable amount of time. 

1 91 . The method of claun 90 wherein said fingerprints are computed in part 

2 from said variable amounts. 

1 92. A method for recognizing a media sample, comprising: 

2 for each of a plurality of media files, providing a file representation of said media file; 

3 providing a sample representation of said media sample; and 

4 identifying at least one similar file representation among said file representations, 

5 wherein said similar file representation is similar to said sample representation, by 

6 searching said file representations, wherein said searching is performed in part m 

7 dependence on a probability of identification of said file representations. 

1 93. The method of claim 92 wherein said at least one similar file representation 

2 exceeds a threshold similarity to said sample representation. 

1 94. The method of claim 92 wherein said identifying step comprises searching a first 

2 subset of said file representations, wherein said first subset contains file 

3 representations having a higher probability of identification than file 

4 representations that are not in said first subset. 
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1 95. The method of claim 94, further comprising searching a second subset of 

2 said file representations if said first subset does not comprise said at least one 

3 similar file representation. 

1 96. The method of claim 92, further comprising ranking said file representations by 

2 said probability of identification, wherein said identifying step comprising 

3 searching said file representations in order of said ranking. 

1 97. The method of claim 96, further comprising termmating said search when 

2 said at least one similar file representation is identified. 

1 98. The method of claim 92 wherein said probability of identification is computed in 

2 part in dependence on a recency of previous identification. 

1 99. The method of claim 98 wherein a recency score of a particular file 

2 representation is increased when said particular file representation is 

3 identified. 

1 100. The method of claim 98 wherein recency scores of said file representations 

2 are decreased at regular time intervals. 

1 101. The method of claim 100 wherein said recency scores are decreased 

2 exponentially in time. 

1 102. The method of claim 92 wherein said probability of identification is computed in 

2 part in dependence on a frequency of previous recognition. 

1 ] 03. A method for recognizing a media sample, comprising identifying media files for which 

2 locations of a substantial plurality of equivalent features of said media files and said 

3 media sample are substantially hnearly related. 

1 1 04. A method for comparing an audio sample and an audio file, comprising: 

2 for each of at least one audio file, computing a plurality of file fingerprints representing 

3 said audio file; 
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4 computing a plurality of sample fingerprints representing said audio sample; and 

5 identifying said audio sample and said audio file if at least a threshold number of said 

6 file fingerprints are equivalent to said sample fingerprints; 

7 wherein said sample fmgerprints are invariant to time stretching of said audio sample. 

1 105. The method of claim 104 wherein said sample fingerprints comprise quotients of 

2 fi:equency components of said audio sample. 

1 106. The method of claim 104 wherein said sample fingerprints comprise products of 

2 firequency components of said audio sample and time differences between points 

3 in said audio sample. 
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